The present inventions relate generally to image processing in a wide-angle video camera, and more specifically to processing image data for transformation into a panoramic view image data stream.
Note that the points discussed below may reflect the hindsight gained from the disclosed inventions, and are not necessarily admitted to be prior art.
Video surveillance systems using wide-angle optical systems apply image data processing to enhance or obscure visual information as necessary using program algorithms. It is useful to be able to define the extents within which image processing operations should take place; for example moving region detection and object tracking software may define the bounding box of a suspicious object and use it to direct enhancement processing to the appropriate area of the image. Regions of interest (ROIs) can be identified in the image based on motion, color, behavior, or object identification. Computer algorithms and programs can be implemented at the camera or at a monitoring base station to automate selection of the ROIs, motion tracking, sound an alarm, summon emergency responders, activate responsive systems (e.g., close doors, seal bulkhead, lock doors, activate ventilation or fire-suppression system, activate lighting, etc.), or other actions as programmed into the system. Further, the object tracking, or alarm, or other processing can be performed on the corrected or on uncorrected versions of the images or video.
In many surveillance systems, standard pan-tilt-zoom (PTZ) cameras capture images of an area. A PTZ camera relies on mechanical gears to adjust the pan, tilt, and zoom of the camera. These cameras have various disadvantages or limitations in surveillance system and applications. Typically, adjusting to concentrate on a region of interest (ROI) necessarily requires a PTZ camera to adjust its view to the ROI and lose sight of the overall area. PTZ cameras are also prone to mechanical failure, misalignment during maintenance, and are relatively heavy and expensive.
In more recent surveillance systems, a new class of cameras replaces the mechanical PTZ mechanisms with a wide-angle optical system and image processing circuitry and software. This type of camera monitors a wide-angle field of view and selects ROIs from the view to transmit to a base station; in this way it emulates the behavior of a mechanical PTZ camera. The wide-angle view can be as much as 180° and capture a full hemisphere of visual data. The wide-angle optics (such as a fisheye lens) introduces distortion into the captured image and processing algorithms are used to correct the distortion and convert it to a view that has a similar view and function as a mechanical PTZ camera. The movements and zoom function are emulated by image data processing accomplished by internal circuitry.
However, these innovations can be used in a camera with a view angle considerably less than 180°. The inventions can be used with annular lenses that often provide imagery that is not from 0 to 90 degree in the tilt (phi in spherical coordinates), but perhaps 40 to 130 degrees. Another lens type can use an ovalized rectangle shape. References to wide-angle cameras include lens systems that meet these criteria.
The captured 3D world space hemispherical image from a wide-angle camera appears distorted, and it can comprise what is sometimes referred to as a fisheye view. Image processing algorithms can correct this view to a more conventional 2D depiction on a monitor and also emulate the movement of a PTZ camera. The wide-view camera can simultaneously focus on multiple ROIs while still able to monitor the area within its entire field of view. This type of camera may be used to improve the processing of selected ROIs, because a wide field is always captured by the camera, and there is no need for mechanical movement and adjustment of the camera's lens system.
The image processing is typically performed on digital image data. The captured image data is converted into a digital format for image data processing at the camera. The processed data can than be transmitted to a base station for viewing. The image data can be converted to an analog signal for transmission to the base station, or the image data can be left in the processed digital format. Additionally, the input signal to the base station, either analog or digital, can be further processed by the base station. Again, image processing and ROI selection at the camera can be fully automatic or subject to various control inputs or overrides that effect automatic image processing. Manual, automatic, or a combination of control options utilizing the combination of the base station and internal camera circuitry can be implemented.